On the Subjectivity of Human Authored Summaries
نویسندگان
چکیده
We address the issue of human subjectivity when authoring summaries, aiming at a simple, robust evaluation of machine generated summaries. Applying a cross comprehension test on human authored short summaries from broadcast news, the level of subjectivity is gauged among four authors. The instruction set is simple, thus there is enough room for subjectivity. However the approach is robust because the test does not use the absolute score, relying instead on relative comparison, effectively alleviating the subjectivity. Finally we illustrate the application of the above scheme when evaluating the informativeness of machine generated summaries.
منابع مشابه
Relative evaluation of informativeness in machine generated summaries
This paper is concerned with the relative evaluation of information content in summaries. We study the effect of crossing the summary-question pairs for a comprehension test based summary evaluation. Using the scheme, machine generated and human authored summaries from the broadcast news stories are evaluated. The approach does not use absolute scores. Instead it relies on a relative comparison...
متن کاملThe Psychological Element in the Phenomenological Analysis of Human Subjectivity
The aim of this paper is to depict the psychological element in phenomenological analysis of the human subjectivity. This analysis should be considered within the philosophical context provided by philosophers of 18th and 19th centuries, and very specifically by Kant as his transcendental philosophy was a turning point for later debates on the human subjectivity. In this paper, I will consider ...
متن کاملIntroducing a Corpus of Human-Authored Dialogue Summaries in Portuguese
In this paper, we introduce a corpus of human-authored dialogue summaries collected through a web-experiment. The corpus features (i) one of the few existing corpora of written dialogue summaries; (ii) the only corpus available for dialogue summaries in Portuguese; and (iii) the only available corpus of summaries produced for dialogues whose participants’ politeness alignment was systematically...
متن کاملpublications and other research outputs Introducing a corpus of human - authored dialogue sum - maries in Portuguese Conference
In this paper, we introduce a corpus of human-authored dialogue summaries collected through a web-experiment. The corpus features (i) one of the few existing corpora of written dialogue summaries; (ii) the only corpus available for dialogue summaries in Portuguese; and (iii) the only available corpus of summaries produced for dialogues whose participants’ politeness alignment was systematically...
متن کاملIntroducing a corpus of human - authored dialogue sum - maries in Portuguese Conference
In this paper, we introduce a corpus of human-authored dialogue summaries collected through a web-experiment. The corpus features (i) one of the few existing corpora of written dialogue summaries; (ii) the only corpus available for dialogue summaries in Portuguese; and (iii) the only available corpus of summaries produced for dialogues whose participants’ politeness alignment was systematically...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Natural Language Engineering
دوره 15 شماره
صفحات -
تاریخ انتشار 2005